Skip to content

Reads treat values in dropped blob files as deleted - #15649

Open
mikechuangmeta wants to merge 2 commits into
meta:mainfrom
mikechuangmeta:export-D131110165
Open

mikechuangmeta wants to merge 2 commits into
meta:mainfrom
mikechuangmeta:export-D131110165

Conversation

@mikechuangmeta

Copy link
Copy Markdown
Contributor

Summary:
Changes

  • Reads apply the compaction rule from D131013279 before fetching. In GetContext::SaveValue (Get, MultiGet, GetEntity, GetEntityLazy, GetMergeOperands, merges) and in DBIter, a value in a dropped blob file reads as a deletion at its sequence number; a wide entity reads as its surviving columns.
  • MissingBlobRepairIterator and reads share IsDroppedBlobIndex and RemoveBlobColumns.
  • New ticker BLOB_DB_DROPPED_BLOB_READ_COUNT. Unrecorded missing blob files still return Corruption.

Why?

  • Reads returned Corruption for values in dropped blob files until a compaction rewrote the key, so the same key read differently before and after compaction. A missing blob file is lost data, not corruption. Reads now return what compaction will produce, including older versions under snapshots, merges over the lost value, and user-defined timestamps.

Changes visible to existing users

  • None while no dropped blob files are recorded: reads add one branch per blob or entity entry, and VersionBlobFetcher construction checks the Version's empty dropped set.
  • New public ticker BLOB_DB_DROPPED_BLOB_READ_COUNT, C++ only because the Java ticker byte space is full.

Known issues, for separate diffs on master

  • Tailing iterators fetch blob values through a Version they do not ref. This diff's own check uses a copy of the dropped list, so it is not affected.
  • GetLatestSequenceForKey merges SST merge operands into null outputs, which crashes for any base value. A dropped base now takes the same path as a deletion.

Design doc

Differential Revision: D131110165

Summary:

Changes
- Every compaction, automatic or manual, whose inputs may reference a blob
  file recorded by `best_efforts_blob_db_repair` runs
  `MissingBlobRepairIterator` in a new mode limited to recorded blob files.
  Lost values become deletions at their sequence number; wide entities keep
  surviving columns.
- These compactions run locally and skip garbage for recorded blob files.
  Unrecorded missing blob files still fail with Corruption.

Why?
- After D131013277 records a dropped blob file, a compaction that drops one of
  its values still fails: garbage for a blob file absent from the Version is
  rejected ("unknown blob file"), and blob GC, filters and merges read the
  missing payload. Reusing the `CompactRangeOptions` repair from D128836898
  lets compactions proceed and removes the dangling references over time.

Changes visible to existing users
- None while no dropped blob files are recorded: each compaction adds an O(1)
  check of its input Version's empty dropped set.

Design doc
- https://mdoc.internalmeta.com/doc/ffcd44f2-3147-4ced-aee4-c5623ccd6d8e

Reviewed By: xingbowang

Differential Revision: D131013279
Summary:
Changes
- Reads apply the compaction rule from D131013279 before fetching. In `GetContext::SaveValue` (Get, MultiGet, GetEntity, `GetEntityLazy`, `GetMergeOperands`, merges) and in `DBIter`, a value in a dropped blob file reads as a deletion at its sequence number; a wide entity reads as its surviving columns.
- `MissingBlobRepairIterator` and reads share `IsDroppedBlobIndex` and `RemoveBlobColumns`.
- New ticker `BLOB_DB_DROPPED_BLOB_READ_COUNT`. Unrecorded missing blob files still return Corruption.

Why?
- Reads returned Corruption for values in dropped blob files until a compaction rewrote the key, so the same key read differently before and after compaction. A missing blob file is lost data, not corruption. Reads now return what compaction will produce, including older versions under snapshots, merges over the lost value, and user-defined timestamps.

Changes visible to existing users
- None while no dropped blob files are recorded: reads add one branch per blob or entity entry, and `VersionBlobFetcher` construction checks the Version's empty dropped set.
- New public ticker `BLOB_DB_DROPPED_BLOB_READ_COUNT`, C++ only because the Java ticker byte space is full.

Known issues, for separate diffs on master
- Tailing iterators fetch blob values through a Version they do not ref. This diff's own check uses a copy of the dropped list, so it is not affected.
- `GetLatestSequenceForKey` merges SST merge operands into null outputs, which crashes for any base value. A dropped base now takes the same path as a deletion.

Design doc
- https://mdoc.internalmeta.com/doc/ffcd44f2-3147-4ced-aee4-c5623ccd6d8e

Differential Revision: D131110165
@meta-cla meta-cla Bot added the CLA Signed label Nov 11, 2026
@meta-codesync

meta-codesync Bot commented Nov 11, 2026

Copy link
Copy Markdown

@mikechuangmeta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D131110165.

@github-actions

Copy link
Copy Markdown

✅ clang-tidy: No findings on changed lines

Completed in 669.5s.

@github-actions

Copy link
Copy Markdown

✅ Claude Code Review

Auto-triggered after CI passed — reviewing commit f0b50b6


Summary

Well-structured change that makes reads consistent with compaction for dropped blob files. The design correctly handles Get, MultiGet, GetEntity, iterators, merges, wide-column entities, and timestamps. Test coverage is comprehensive.

High-severity findings (0):
No high-severity findings.

Full review (click to expand)

Findings

🟡 MEDIUM

M1. EmbeddedAwareBlobFetcher inherits storage_with_dropped_blob_files_ but base fetcher's value is not proxied — db/blob/blob_fetcher.h
  • Issue: storage_with_dropped_blob_files_ is placed on the BlobFetcher base class. EmbeddedAwareBlobFetcher inherits from BlobFetcher and decorates a base fetcher. If GetContext::blob_fetcher_ ever pointed to an EmbeddedAwareBlobFetcher, storage_with_dropped_blob_files() would return nullptr (the default) instead of the underlying VersionBlobFetcherBase's value, silently disabling the dropped blob check.

    In the current code, GetContext::blob_fetcher_ always points to a VersionBlobFetcherBase subclass, so this works. But the design is fragile.

  • Suggested fix: Move storage_with_dropped_blob_files() and the protected member to VersionBlobFetcherBase, or add a comment on the base class documenting why it lives there (e.g., to avoid dynamic_cast in GetContext).

M2. remaining_columns lifetime in GetContext::SaveValue is fragile — table/get_context.cc
  • Issue: remaining_columns is a stack-local std::string. After CheckDroppedBlobReferences returns kRemainingColumns, unpacked_value is reassigned to point into it. The Slice is consumed within the same function scope, so this is correct today. But the lifetime dependency between remaining_columns and unpacked_value is implicit and easy to break in future refactoring.

  • Suggested fix: Consider adding a brief comment noting that unpacked_value may alias remaining_columns and both must stay alive through the switch.

M3. Double PrepareValueInternal() on the dropped-blob-check path in FindNextUserEntryInternal — db/db_iter.cc
  • Issue: The diff adds PrepareValueInternal() for blob/entity entries before the switch. The existing switch case for kTypeBlobIndex/kTypeWideColumnEntity also calls PrepareValueInternal(). When the type is NOT converted to deletion (blob file is not dropped), the value is prepared twice. PrepareValueInternal is a no-op when already prepared, so this is safe but adds one extra function call + branch on the (rare) path where dropped blob files exist but the specific entry's blob file is NOT dropped.

  • Suggested fix: Acceptable as-is. The UNLIKELY(!dropped_blob_files_.empty()) guard ensures this only runs when dropped files exist. No change needed.

🟢 LOW / NIT

L1. Feature-off cost: 24 bytes per DBIter + 8 bytes per BlobFetcher — db/db_iter.h, db/blob/blob_fetcher.h
  • Issue: dropped_blob_files_ adds 24 bytes to every DBIter. storage_with_dropped_blob_files_ adds 8 bytes to every BlobFetcher. In the common case (no dropped files), these are zeros. Per the "feature-off cost is zero" principle, this is a small constant cost. The DBIter copy is needed for tailing iterator safety; the BlobFetcher pointer avoids repeated checks.

  • Suggested fix: The trade-off is reasonable and well-motivated. Consider a brief comment on dropped_blob_files_ explaining why a copy (not a pointer) is needed.

L2. appendToReplayLog captures original type before conversion — table/get_context.cc
  • Issue: The replay log at line 430 captures the original parsed_key.type and value before the dropped blob conversion. For transaction conflict detection this is correct (the log records what exists in storage). Worth noting for future readers.

  • Suggested fix: None needed.

L3. New ticker not in Java — include/rocksdb/statistics.h
  • Issue: PR description notes "C++ only because the Java ticker byte space is full." This is a known limitation.

  • Suggested fix: Already documented.

L4. InstallCompactionResults condition change — db/compaction/compaction_job.cc:2765
  • Issue: The old condition was compaction->skip_missing_blob_files() && ... && !storage_info->GetBlobFileMetaData(blob_file_number). The new condition is (compaction->skip_missing_blob_files() || storage_info->IsDroppedBlobFile(blob_file_number)). The old code additionally checked that the blob file is NOT in the Version's metadata. For skip_missing_blob_files, this extra check is now dropped. However, skip_missing_blob_files is set when best-efforts recovery drops blob files entirely, so they won't be in the Version either. For the dropped-blob-file case, IsDroppedBlobFile is the right check since dropped files are in the Version's metadata but marked as dropped. The logic is correct.

  • Suggested fix: None needed.

Cross-Component Analysis

Context Affected? Assumptions hold? Notes
WritePreparedTxnDB YES YES Visibility via read_callback_ is independent of dropped blob checks
ReadOnly DB YES YES Covered by test BestEffortsBlobDbRepairReadOnly
Tailing iterators YES YES dropped_blob_files_ copied in constructor; test covers Version release
User-defined timestamps YES YES Uses kTypeDeletionWithTimestamp; timestamp history test included
CompactionService YES YES RepairsMissingBlobReferences() forces local compaction
Legacy BlobDB N/A YES dropped_blob_files_ empty when expose_blob_index is true
GetLatestSequenceForKey Partial YES SST lookup path doesn't pass blob_fetcher, so dropped blob entries are found as blob indexes (pre-existing behavior, noted in PR as known issue)
Concurrent writers YES YES dropped_blob_files_ is immutable once Version is created

Positive Observations

  • The design reuses IsDroppedBlobIndex across reads and compaction, ensuring reads return what compaction will produce.
  • RemoveBlobColumns consolidates duplicated entity column filtering into a single reusable function with a clean callback interface.
  • Copying dropped_blob_files_ for tailing iterators cleanly avoids Version lifetime issues.
  • Comprehensive test coverage: 10 new test cases covering Get, MultiGet, iterators (forward, reverse, tailing, unprepared), merges, wide-column entities, timestamps, auto-compaction, GC, read-only, and blob direct write.
  • UNLIKELY guards minimize feature-off cost on hot paths.
  • RepairsMissingBlobReferences() efficiently checks only whether any input SST's oldest blob file could be in the dropped set, using the sorted property.

ℹ️ About this response

Generated by Claude Code.
Review methodology: claude_md/ci_review_prompt.md

Limitations:

  • Claude may miss context from files not in the diff
  • Large PRs may be truncated
  • Always apply human judgment to AI suggestions

Commands:

  • /claude-review [context] — Request a code review
  • /claude-query <question> — Ask about the PR or codebase

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant